This Page Is Inserted by IFW Operations 
and is not a part of the Official Record 

BEST AVAILABLE IMAGES 



Defective images within this document are accurate representations of the 
original documents submitted by the applicant. 

Defects in the images may include (but are not limited to): 



BLACK BORDERS 



TEXT CUT OFF AT TOP, BOTTOM OR SIDES 



FADED TEXT 



ILLEGIBLE TEXT 



SKEWED/SLANTED IMAGES 



COLORED PHOTOS 



BLACK OR VERY BLACK AND WHITE DARK PHOTOS 



GRAY SCALE DOCUMENTS 



IMAGES ARE BEST AVAILABLE COPY. 



As rescanning documents will not correct images, 
please do not report the images to the 
Image Problems Mailbox. 





Saltamar 

Innovations 




30 Fern lane 

South Portland, ME 04106 
USA 

Phone: +1(207) 799-9733 
Fax:+1(207) 228-3694 
E-maii: Shalom@Saltamar.com 
HTTP://www.saltamar.com 



(9/ 



In the United States Patent and Trademark Office 



re application of: Pekka Ala Honkola 



For: Adaptive Media Stream 



Serial No./Confirm. 



09/974,589 



Filed on: 



October 10, 2001 



Correspondence Date 11/26/2001 



Group: 



Examiner: 



Technok 



Docket: 



0138US-Oplayo 



2 2001 



^enter2100 



Priority Document Filing in accordance with 37 CFR 1.55 

Commissioner of Patents 
Washington DC, 20231 

Sir, 

Applicant hereby requests to enter into record the enclosed certified copy of Finnish patent 
application #2001 1856, filed September 20, 2001. Applicant claims the benefit of priority to 
this application in accordance with 35 USC 119. 




Respectfully submitted 
AS- 

Shalom Wertsberger 
30 Fern Lane 

South Portland, ME 04106 
Phone: (207) 799-9733 
Fax: (207) 799-3698 

Agent for Applicant 
Reg. Num. 43,359 



Certificate of Mailing Under 37 CFR 1.10: 1 hereby certify that this correspondence 
is being deposited with the United States Postal Service with sufficient postage as 
express mail, No. EJ680975756US, addressed to: Commissioner for Patents, 
Washington, DC 20231. 



on: By : 



Signature 




PATENTTI- JA REKI STERI HALLI TUS 

NATIONAL BOARD OF PATENTS AND REGISTRATION 



Helsinki 




ETUOIKEUSTODISTUS 
PRIORITY DOCUMENT 




Haki j a 
Applicant 



Patentt ihakemus nro 
Patent application no 



Oplayo Oy 
Helsinki 



20011856 



Tekemispaiva 
Filing date 



20. 09.2001 



Kansainvalinen luokka 



G06F 



International class 

Keksinnon nimitys 
Title of invention 

"Adaptive media stream" 
(Adaptiivinen mediavirta) 



Taten todistetaan, etta oheiset asiakirjat ovat tarkkoja jaljennoksia 
patentti- ja rekisterihallitukselle alkuaan annetuista selityksesta , 
patenttivaatimuksista, tiivistelmasta ja piirustuksista . 

This is to certify that the annexed documents are true copies of the 
description, claims, abstract and drawings originally filed with the 
Finnish Patent Office. 



Pirjo Kaila 
Tutkimussihteerf 




Maksu 
Fee 



300,- mk 
300,- FIM 



Osoite: Arkadiankatu 6 A Puhelin: 09 6939 500 Telefax: 09 6939 5328 

P.O.Box 1160 Telephone: + 358 9 6939 500 Telefax: + 358 9 6939 5328 

FIN-00101 Helsinki, FINLAND 



20/09 01 TO lo:4S *AI 3o$ 9^801305 COMPATENT Hjf PRH KIRjI 



PRH KIRJAAMO @002 



1 

L / 



Adaptive media stream 



Field of the Invention 

This invention relates to media streams transmitted in a network, 
5 such as the Internet. Especially, the invention relates to media streams con- 
cerning video pictures and voice. 



Background of the Invention 4 

When media streams that contain continuous data, for example 

10 video pictures and voice, are transmitted in a network, such as the Internet, 
the speeds of the media streams have to be adjusted with the speed of the 
transmission connection available at the time. Often, in the case of video pic- 
tures, the adjustment is made by changing the quality of the pictures. The 
speed of a transmission connection in the Internet depends strongly on a 

15 user's Internet terminal. Some terminals support technically higher speed 
rates than others (usually older terminals). Thus, speed adjusted media 
streams must be offered for a variety of terminals. 

Transmission speed may also vary with time. For example, when 
congestion happens in the network, the transmission speed may decrease 

20 considerably. After the congestion, the transmission speed returns to the 
normal speed rate. 

Nowadays, most of the content of the Internet is In WWW servers, 
which use an HTTP protocol for transmitting desired contents to users. 
Originally, the HTTP protocol was not designed for continuous media 

25 streams (video, voice). The problem of the known solutions is that when a 
media stream has to be adjusted for a changed transmission condition, i.e. to 
be an adaptive media stream, a special software or protocol must be used, 
instead of the HTTP protocol. Real Server, Quicktime Server, and Windows 
Media Server are commercial server software, which support adaptive media 

30 streams. These software use special protocols, such as RTP (Real-Time 
Transport Protocol) and RTSP (Real Time Streaming Protocol), which are 
designed for the adaptive media streams. 

However, using these software usually requires a user to invest 
and maintain parallel devices and software. If the user is a community, such 

35 as a company, support staff must be educated to use these systems. For a 
service provider this means in practice that a special media stream server 
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must be maintained. The goal of this invention is to eliminate these draw- 
backs. This is achieved in a way described in the claims- 
Summary of the Invention 
5 The idea of the invention is to change the source of an adaptive 

media stream when an available transmission speed rate changes. Each 
source contains essentially the same information (such as video and voice), 
but the suitability of each source for transmission has been adjusted to a cer- 
tain speed rate. The coding of the source information can be different among 

10 the sources, which affects the speed rate. The picture information can be 
black-and-white instead of a color picture. The picture size can be smaller or 
larger. The video can be with or without voice. When the source is changed, 
also a new stream is created for the new source. In other words, the adaptive 
media stream comprises at least one medi^ stream for transmitting informa- 

15 tion. 

In an arrangement according to the invention a file, which contains 
sources for different media streams, is formed. At the beginning of the trans- 
mission the optimal transmission speed rate at the time is chosen. The 
source in the file, which corresponds to the available speed rate, is selected 

20 to be the source of the adaptive media stream. When the available transmis- 
sion speed rate changes, either to be faster or slower, a command for chang- 
ing the source of the adaptive media stream, is sent from the user's terminal 
to the service provider's server wherein the file of the service is. Commands 
are sent using an HTTP protocol. Wheh the available transmission speed 

25 rate changes again, a new command for bhanging the source is sent from the 



user's terminal to the service provider's 
sent when it is possible to use a higher 
rate must be used. 



server. The commands are always 
speed rate or when a lower speed 



i 

30 Brief Description of the Drayririgs 

In the following the invention described in more detail by means 

of FIGs 1 - 3 in the attached drawings wHere. 

i 
j 

FIG. 1 illustrates an example of the structure of a file according to the in- 
35 vention, j 
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FIG. 2 illustrates an example of a flow chart describing the function of the 
inventive arrangement, 

FIG. 3 illustrates an example oft an arrangement according to the inven- 
tion. 

5 

Detailed Description of the Invention 

FIG. 1 describes an example of a file structure according to the in- 
vention. The file is in the service providers server from where users can 
download it. Let's say the file contains a video in two different formats. At 

1 0 the beginning of the file there exists a header 1 , which contains starting bytes 
for two different media sources - the sources for the media stream. Source 1, 
i.e. stream 1 is a 320*240 video file with a speed rate 200 kbps 2. The start- 
ing byte 4 in the file is 2000. Source 2, i.e. stream 2 is a 160*120 video file 
with a speed rate 40 kbps 3. The starting byte 5 in the file is 800000. If the 

15 network and a user's terminal support the transmission speed rate 200 kbps 
then stream 1 is sent to the user - otherwise stream 2 is sent. Let's assume 
that 40 kbps is always available. 

Stream 1 is sent to the user. The stream contains timing marks 6, 
7, 8, 9, frequently indicating the byte of the file where the stream is running at 

20 the time and the corresponding byte 6A f 7A, 8A, 9A. in stream 2. If for some 
reason, the network can not transmit stream 1 with 200 kbps, or the user's 
terminal can not receive at this speed rate, the transmission of the video is 
changed to stream 2. For example, when the need to change the speed rate 
to a lower one is noticed and the next timing mark is, let's say, the mark 7 in- 

25 dicating time 1000 ms, it tells the corresponding byte 805000 in stream 2 to 
where the source of the transmission of the video transfers 1 1 . 

In the same way, stream 2 contains timing marks 10, 11, 12, 13, 
frequently indicating the byte of the file where the stream is running at the 
time and the corresponding byte 10A, 11A, 12A, 13A, in stream 1. When the 

30 capability of transmitting 200 kbps is noticed during the transmission at 40 
kbps rate, the transmission of the video is changed to stream 1. For example, 
when the availability of the higher speed rate is noticed and the next timing 
mark is the mark 13 indicating time 3000 ms, it tells the corresponding byte 
77000 in stream 2 to where the source of the transmission of the video trans- 

35 fers 9, 
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FIG. 2 shows an example of a flow chart, which illustrates a func- 
tion and method of the invention. FIG. 3 shows an example of the inventive 
arrangement. When a user visits a home page of a service provider, he/she 
may find something he wants to order, such as a video show. The user sends 
5 34 a request message from his terminal 33 to the service provider's server 31 
through the network 32, such as the Internet, to get the show using a GET 
method that is one of the methods, which belongs to the HTTP1.1 protocol. 
Let the request be: 

G ET http ://www.serviceprovid er . com/video/m us ic/ba nd 3 . htm t . 

10 HTTP/1.1 

Mostly, the transmission speed rate depends on the available (and 
optional) processing speed of the terminal, ^so it is also sent a list of alterna- 
tive speed rates from the user's terminal. The server chooses 21 one of the 
supported speed rate from the list. (It should be noticed that the server can 

15 be informed of the capability of a terminal in another way as well.) In this 
case, the user's terminal supports speed rates of 200 kbps and 40 kbps. Let's 
assume that the desired file 36 is the file in FIG. 1. If the speed rate of 200 
kbps 22 is available, stream 1 is selected 24. Otherwise, the speed rate of 40 
kbps 23 is chosen and stream 2 is selected 25. (Notice that the higher speed 

20 rate means the higher bandwidth.) 

In the selection of the speed rate of 200 kbps the server starts to 
play 26 stream 1 from the right byte (2000) of the file. The data and the tim- 
ing marks are sent 35 to the user's terminal during the play. The terminal 
checks 28 frequently that the bandwidth is okay for the selected stream. If the 

25 bandwidth is OK, playing stream 1 is continued. If the bandwidth is too fast at 
any given time, stream 2 is selected 210 and stream 2 starts to play 27, 311 
from the right byte of the file defined by the latest timing mark. The bandwidth 
of stream 2 is checked 29 similarly as the bandwidth of stream 1 , and when 
the bandwidth is again available for the speed rate of 200 kbps, stream 1 is 

30 selected 211. 

In the selection of the speed rate of 40 kbps the server starts to 
play 25 stream 2 from the right byte (800000) of the file. The data and the 
timing marks are sent 35 to the user's terminal during the play. The terminal 
checks 29 frequently that the bandwidth is okay for the selected stream. If the 

35 bandwidth is OK, playing stream 2 is continued. If the bandwidth is too slow 
at any given time, stream 1 is selected 211 and stream 1 starts to play 26, 
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and is sent 31 1 to the terminal 33 from the right byte of the file defined by the 
latest timing mark. The bandwidth of stream 1 is checked 28 similarly as the 
bandwidth of stream 2, and when the bandwidth is again too fast for the 
speed rate of 200 kbps, stream 2 is selected 210. 
5 The checking of the bandwidth of the streams can be made, for 

example, measuring a filling degree of a buffer for incoming data in the user's 
terminal. When the buffer is becoming empty, the process is too fast com- 
pared to what is needed, and the speed rate can be dropped. When the 
buffer is becoming full, the process is too slow compared to what it can be, 

10 and the speed rate can be raised. After the measurement, the suitable 
stream is selected 210, 211 and a GET command with an additional field 
RANGE is sent 310 from the user's terminal to the server for changing the 
stream. For example, let the GET command be GET 
http://www.serviceprovider.com/video/music/band3.html RANGE: bytes= 

15 805000- when the transmission speed rate changes from 200 kbps to 40 
kbps. 

It is worth noting that when a stream is changed, a new whole 
stream (instance) is created and the old stream is terminated after the new 
stream starts playing. From the view of the user the desired file, such as a 

20 video show, seems to continue without any break thanks to the new stream 
starting to play from the right byte indicated by the timing mark. So, when the 
source is changed, also a new stream is created for the new source. In other 
words, the adaptive media stream comprises at least one media stream for 
transmitting information. Furthermore, it may be said that the adaptive media 

25 stream comprises several (or one if the speed rate remains the same during 
the whole transmission) instances (separate streams) of the same object (the 
adaptive media stream containing information such as video). Naturally, 
there are more ways to model the adaptive media stream, but common for 
these models are that transmission of information happens in the adaptive 

30 stream and a created stream under control of the adaptive stream handles 
the transmission of the information of a specific source. 

For creating the inventive arrangement means 39 for choosing an 
original transmission speed rate and selecting a suitable source in a service 
provider's server is needed. In a user's terminal means 37 for checking the 

35 suitability of the transmission speed rate and means 38 for reselecting a suit- 
able source are needed. 
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Although, the processing speed of a user's terminal is mostly the 
decisive element when selecting a suitable transmission speed rate, it may 
happen that the network becomes congested. In congestion, the available 
speed rate decreases and the transmission speed rate must be lowered. On 
5 the other hand, if the network allows a higher speed, the transmission speed 
rate may be increased. 

Using the inventive arrangement an adaptive media stream can 
be created wfth a normal HTTP server. Therefore, there is no need for the 
use of a special media stream server, or parallel devices and software. Re- 
10 markable cost savings are achieved, since usually service providers and us- 
ers support the HTTP protocol. Also the reliability of the system increases 
because of fewer sources of faults. 

It is clear that inventive arrangement can be implemented in many 
ways. For example, the time period between two timing marks can be differ- 
15 ent than in the example of this text. It can be any suitable period depending 
on the features of a solution, such as 2 or 4 seconds. The format of the in- 
ventive file can be any suitable format as well, such as MPEG or another. 

The inventive file may comprise more than two different streams 
(containing the same content), from which the most suitable stream is se- 
20 lected at any given time. Therefore, timing marks must contain a number of 
references for the corresponding byte in the other data sources. 

All the sources do not have to be in the same file, but each source 
can be a separate file. However, the use of this kind of structure is not as ef- 
fective in the light of using as little processing power as possible compared to 
25 the use of one file structure. In the structure of several files, timing marks 
must indicate the right file with the corresponding byte. 

Although, a sending terminal have been described to be a server 
in the examples above, it is clear that the sending terminal can be another 
terminal capable to send information. 
30 According to the above-mentioned matters, it is evident that the 

invention is not restricted to the solutions described in this text, but it can be 
modified for various solutions, in the scope of the inventive idea. 
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Claims 

1. A method for providing an adaptive media stream between a 
sending terminal and a user's terminal characterized in that the 
adaptive media stream comprises at least one media stream and the method 

5 comprises the steps of: 

a) choosing a transmission speed rate for the media stream, 

b) selecting a suitable data source for the media stream among 
sources containing essentially the same information, each source 
intended for a certain transmission speed rate, 

10 c) playing the media stream, 

d) checking the suitability of the tran%mission speed rate : 

e) either continuing playing the media stream if the transmission 
speed is still suitable or, 

f) selecting a new data source, which is suitable at the moment, 

15 g) as a response for the selection of the new data source, playing a 

new media stream, 
h) repeating steps d) to g) until the stream, which is played, stops. 

2. A method according to claim 1 characterized in that 
step b) comprises the phase of reading a header field of a file, which contains 

20 at least two data sources, the header indicating the starting points of the 
sources and to which transmission speed rate each source is indended. 

3. A method according to claim 2 characterized in that 
step c) comprises the phase of starting to read from the starting point of the 
selected source. 

25 4. A method according to claim 1 or 3 characterized in 

that step d) comprises the phase of monitoring a filling degree of a buffer for 
incoming data in the user's terminal. 

5. A method according to claim 4 characterized in that if 
the buffer is becoming empty or full, then step f) is chosen, else step e) is 

30 chosen. 

6, A method according to claim 1 or 5 characterized in 
that the information of the selection of the new data source is transmitted 
from the user's terminal to the sending terminal using a GET method of an 
HTTP protocol with an optional RANGE field. 

35 7. A method according to claim 1 characterized in that 

step g) comprises phases of terminating the old media stream after starting 
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to play the new media stream and creating the new media stream before 
playing the new media stream. 

8. A method according to claim 1 or 7, characterized in 
that step g) further comprises the phase of reading timing marks in the data 

5 source for defining the right point to start playing the new media stream. 

9. A method according to claim 1 characterized in that 
step b) comprises the phase of reading a header field of files, where each 
contains one data source, the header indicating the starting points of the 
source and to which transmission speed rate the source is indended. 

10 10. An arrangement for providing an adaptive media stream be- 

tween a sending terminal and a user's terminal, characterized in that 
the adaptive media stream comprises at least one media stream and the ar- 
rangement comprises: 

a) means in the sending terminal for choosing a transmission speed 
1 5 rate for the media stream, 

b) a data structure in the sending terminal comprising at least two 
data sources containing essentially the same information, each 
source intended for a certain transmission speed rate, 

c) means in the sending terminal for selecting a suitable data source 
20 from among the sources for the media stream, 

d) means in the user's terminal for checking the suitability of the 
transmission speed rate when the media stream is played, 

e) means in the user ! s terminal for teselecting a suitable data source 
from among the sources for a new media stream. 

25 11 . An arrangement according to claim 10, characterized 

in that the arrangement further comprises means for playing the data struc- 
ture in a way that the playing is started from the right point of the data struc- 
ture. 

12. An arrangement according to claim 11, characterized 
30 in that means a) comprises means for reading a message from the user's ter- 
minal, which defines alternative transmission speed rates. 

13. An arrangement according to claim 11 T characterized 
in that each source comprises timing marks, which indicate the correspond- 
ing point in the other sources. 

35 14. An arrangement according to claim 13, characterized 

in that the arrangement further comprises a GET method of an HTTP proto- 
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col with an optional RANGE field for transmitting the information of the rese- 
lection of the suitable data source from the user's terminal to the sending 
terminal. 

15. An arrangement according to claim 11 T characterized 
in that means d) comprises means for monitoring a filling degree of a buffer 
for incoming data in the user's terminal. 

16. An arrangement according to claim 11, characterized 
in that the data structure is a file containing the data sources. 

17. An arrangement according to claim 16 T characterized 
in that means c) comprises means for reading a header of the data structure, 
which indicates the starting points of the sources and to which transmission 
speed rate each source is indended. 

18. An arrangement according to claim 11, characterized 
in that the data structure is a number of files, each of them containing one 
data source. 

19. An arrangement according to claim 18, characterized 
in that means c) comprises means for reading headers of the files, which in- 
dicate the starting point of the source and \o which transmission speed rate 
the source is indended. 



20 
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(57) Abstract 

This invention relates to media streams transmitted in a 
network, such as the Internet. The idea of the Invention is 
to change the source of the content of a media stream 
when an available transmission speed rate changes. Each 
source contains essentially the same information (such as 
video and voice), but the suitability of each source for 
transmission has been adjusted to a certain speed rate. 
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2100 
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HEADER 
STREAMS 

1. 320*240 200KBPS2D0D 

2. ISO* 1 2D 40KBPS 800000 



STREAM 1 

TIME 10 MS, STREAM 2 ® BDD10Q 
DATA 

TIME 1000 MS, STREAM 2 @ 805000 
DATA 

TIME 2000 MS, STREAM 2 @ 810000 
DATA 



TIME 3000 MS, STREAM 2 ® B 15000 



DATA 



STREAM 2 

TIME 10 MS, STREAM 1 @ 2100^ 
DATA 



TIME 1 000 MS, STREAM 1 @ 27000 * 
DATA 

TIME 2D00 MS, STREAM 1 @ 52000 
DATA 

nME 3000 MS, STREAM 1 @ 7700D 
DATA 



EA 



*7A 



8A 



9A 



IDA 



11A 



12A 



13A 



fig. 1 



4 



20/09 '01 TO 15:51.FAJ 33-8 



>o » t'ouiaub -> PATREK ASIAK^PALVELU; Sivu 13 

[801305 COMPATENT 



-»->-* PRH KIRJAAMO ©013 




23 



40 KBPS OK 



22 



2DO KBPS OK 



SELECTING 
STREAM 1 



2G 



24 



SELECTING 
STREAM 2 



PLAYING 
STREAM 1 



21D 



SELECTING 
STREAM 2 



25 



27 



PLAYING 
STREAM 2 



SELECTING 
STREAM 1 



211 , r 
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